While electronic documents stored on computers provide a number of advantages over written documents, many users continue to perform some tasks with printed versions of electronic documents. These tasks include, for example, reading and annotating the documents. With annotations, the paper version of the document assumes particular significance because the annotations typically are written directly onto the printed document by the user. One of the problems, however, with directly annotating a printed version of a document is the difficulty in later converting the annotations into electronic form. Ideally, electronically stored annotations should correspond with the electronic version of the document in the same way that the handwritten annotations correspond with the paper version of the document.
This correspondence usually requires the original or another user to wade through the annotations and personally enter them into a computer. In some cases, a user may electronically scan the annotations written on the paper document, thereby creating a new electronic document. These multiple steps make reconciliation between the printed version of a document and the electronic version of the document difficult to handle on a repeated basis. Further, scanned images frequently cannot be edited. Thus, there may be no way to separate the annotations from the original text of the document. This makes using the annotations difficult.
To address this problem, pens have been developed to capture annotations written onto printed documents with the pen. This type of pen includes a camera, which captures images of the printed document as a user writes annotations. With some examples of this type of pen, however, the pen may employ ink that is invisible to the camera. The pen may, for example, employ non-carbon ink and infrared illumination for the camera, which prevents the camera from “seeing” annotation written with the ink. With this type of pen, the pen infers the movement of the pen tip forming the annotations on the document from the images captured by the pen during the writing of the annotations. In order to associate the images with the original electronic document, however, the position of the images relative to the document must be determined. Accordingly, this type of pen often is employed with paper that includes a pattern that uniquely identifies different locations on the paper. By analyzing this pattern, the computer receiving an image can determine what portion of the paper (and thus what portion of the printed document) was captured in the image.
While the use of such patterned paper or other media allows written annotations on a paper document to be converted into electronic form and properly associated with the electronic version of the document, this technique is not always reliable. For example, a document containing text on the paper may obscure areas of the pattern. If the pen captures an image of one of these areas, then the computer may not be able to use the pattern to accurately determine the location of the document portion captured by the image. Instead, the computer must employ an alternate technique to identify the location of the document portion captured in the image. For example, the computer may perform a pixel-by pixel comparison of the captured image with the electronic document.
A pixel-by-pixel comparison will usually identify the portion of document in a captured image, but this technique has a high processing overhead. To perform this technique, for example, a transform of, e.g. rotation, and scale, between the captured image and the document image typically must first be estimated so that the captured image can be warped and matched with the document image pixel-by-pixel. If the transform is unknown, all possible rotations and scales must be considered. Additionally, a reference pixel in the image is selected. Every pixel in the warped image then is compared with a corresponding pixel in the electronic document such that the image reference pixel is compared to a first location in the electronic document. This comparison must then be repeated so that the reference pixel is eventually compared to each location in the electronic document. The comparison with the highest correspondence between the image pixels and the electronic document identifies the position of the reference pixel relative to the electronic document, and thus the portion of the document captured in the image. Accordingly, it would be desirable to provide a technique that allows a computer to determine the location of a portion of a document in a captured image without having to perform a pixel-by-pixel comparison of the image with the entire document.